Diagnostic accuracy and clinical management by realtime teledermatology. Results from the Northern Ireland arms of the UK Multicentre Teledermatology Trial

Diagnostic accuracy and management recommendations of realtime teledermatology consultations using low-cost telemedicine equipment were evaluated. Patients were seen by a dermatologist over a video-link and a diagnosis and treatment plan were recorded. This was followed by a face-to-face consultation on the same day to confirm the earlier diagnosis and management plan. A total of 351 patients with 427 diagnoses participated. Sixty-seven per cent of the diagnoses made over the video-link agreed with the face-to-face diagnosis. Clinical management plans were recorded for 214 patients with 252 diagnoses. For this cohort, 44% of the patients were seen by the same dermatologist at both consultations, while 56% were seen by a different dermatologist. In 64% of cases the same management plan was recommended at both consultations; a sub-optimum treatment plan was recommended in 8% of cases; and in 9% of cases the video-link management plans were judged to be inappropriate. In 20% of cases the dermatologist was unable to recommend a suitable management plan by video-link. There were significant differences in the ability to recommend an optimum management plan by video-link when a different dermatologist made the reference management plan. The results indicate that a high proportion of dermatological conditions can be successfully managed by realtime teledermatology.

The quality of reporting of diagnostic accuracy studies in glaucoma using scanning laser polarimetry

PURPOSE: Scanning laser polarimetry (SLP) has been proposed as a useful diagnostic test for glaucoma. This study was conducted to evaluate the quality of reporting of published studies using the SLP for diagnosing glaucoma. METHODS: A validated Medline and hand search of English-language articles reporting on measures of diagnostic accuracy of the SLP for glaucoma was performed. Two reviewers independently selected and appraised the manuscripts. The Standards for Reporting of Diagnostic Accuracy (STARD) checklist was used to evaluate the quality of each publication. RESULTS: A total of 47 papers were identified of which the first 10 (from 1997 to 2000) and the last 10 articles (from 2004 to 2005) were appraised. Interobserver rating agreement of STARD items was high (85.5% agreement, ?=0.796). The number of STARD items properly reported ranged from 3/25 to 19/25. Only a quarter of studies (5/20) explicitly reported more than half of the STARD items. Important aspects of the methodology were often missing such as participant sampling (reported in 40% of manuscripts), masking of the readers of the index test and reference standard (reported in 20% of manuscripts), and estimation of uncertainty (eg, 95% confidence intervals, reported in 25% of manuscripts). There was a slight increase in the number of STARD items reported with time. CONCLUSIONS: The quality of reporting of diagnostic accuracy tests for glaucoma with SLP is suboptimal. The STARD initiative may be a useful tool for appraising the strengths and weaknesses of diagnostic accuracy studies. © 2007 Lippincott Williams & Wilkins, Inc.

Diagnostic accuracy and reproducibility of tendency oriented perimetry in glaucoma

PURPOSE. To evaluate the diagnostic capability of tendency oriented perimetry (TOP) in glaucoma. METHODS. A): The diagnostic accuracy of mean defect (MD), square-root of the loss variance (sLV), and number of pathologic points (NPP) was calculated in 295 normal and 414 glaucoma eyes (179 early, 112 moderate, and 123 advanced) examined with TOP. B): Threshold fluctuation (F) and its relationship with the loss variance (LV) was measured in 34 normal and 33 glaucoma eyes (mean MD=3 dB; SD=3.9) for TOP and for full-threshold perimetry (FT). C): Twenty-eight eyes with stable glaucoma (mean MD=9.5 dB; SD=7.2) were examined six times to quantify LV error. D): TOP and FT were tested with the simulation program PeriSim using different behavior models. RESULTS. A): The best diagnostic index in early glaucoma (MD

The quality of reporting of diagnostic accuracy studies in glaucoma using the Heidelberg retina tomograph

PURPOSE. Scanning laser tomography with the Heidelberg retina tomograph (HRT; Heidelberg Engineering, Heidelberg, Germany) has been proposed as a useful diagnostic test for glaucoma. This study was conducted to evaluate the quality of reporting of published studies using the HRT for diagnosing glaucoma. METHODS. A validated Medline and hand search of English-language articles reporting on measures of diagnostic accuracy of the HRT for glaucoma was performed. Two reviewers selected and appraised the papers independently. The Standards for Reporting of Diagnostic Accuracy (STARD) checklist was used to evaluate the quality of each publication. RESULTS. A total of 29 articles were included. Interobserver rating agreement was observed in 83% of items (? = 0.76). The number of STARD items properly reported ranged from 5 to 18. Less than a third of studies (7/29) explicitly reported more than half of the STARD items. Descriptions of key aspects of the methodology were frequently missing. For example, the design of the study (prospective or retrospective) was reported in 6 of 29 studies, and details of participant sampling (e.g., consecutive or random selection) were described in 5 of 29 publications. The commonest description of diagnostic accuracy was sensitivity and specificity (25/29) followed by area under the ROC curve (13/29), with 9 of 29 publications reporting both. CONCLUSIONS. The quality of reporting of diagnostic accuracy tests for glaucoma with HRT is suboptimal. The STARD initiative may be a useful tool for appraising the strengths and weaknesses of diagnostic accuracy studies. Copyright © Association for Research in Vision and Ophthalmology.

The quality of reporting of diagnostic accuracy studies published in ophthalmic journals

Aim: To evaluate the quality of reporting of all diagnostic studies published in five major ophthalmic journals in the year 2002 using the Standards for Reporting of Diagnostic Accuracy (STARD) initiative parameters. Methods: Manual searching was used to identify diagnostic studies published in 2002 in five leading ophthalmic journals, the American Journal of Ophthalmology (AJO), Archives of Ophthalmology (Archives), British Journal of Ophthalmology (BJO), Investigative Ophthalmology and Visual Science (IOVS), and Ophthalmology. The STARD checklist of 25 items and flow chart was used to evaluate the quality of each publication. Results: A total of 16 publications were included (AJO = 5, Archives = 1, BJO = 2, IOVS = 2, and Ophthalmology = 6). More than half of the studies (n = 9) were related to glaucoma diagnosis. Other specialties included retina (n = 4) cornea (n = 2), and neuro-ophthalmology (n = 1). The most common description of diagnostic accuracy was sensitivity and specificity values, published in 13 articles. The number of fully reported items in evaluated studies ranged from eight to 19. Seven studies reported more than 50% of the STARD items. Conclusions: The current standards of reporting of diagnostic accuracy tests are highly variable. The STARD initiative may be a useful tool for appraising the strengths and weaknesses of diagnostic accuracy studies.

The Quality of Reporting of Diagnostic Accuracy Studies of Optical Coherence Tomography in Glaucoma

Objective: To evaluate the quality of reporting of diagnostic accuracy studies using optical coherence tomography (OCT) in glaucoma. Design: Descriptive series of published studies. Participants: Published studies reporting a measure of the diagnostic accuracy of OCT for glaucoma. Methods: Review of English language papers reporting measures of diagnostic accuracy of OCT for glaucoma. Papers were identified from a Medline literature search performed in June 2006. Articles were appraised using the 25 items provided by the Standards for Reporting of Diagnostic Accuracy (STARD) initiative. Each item was recorded as full, partially, or not reported. Main Outcome Measures: Degree of compliance with the STARD guidelines. Results: Thirty papers were appraised. Eight papers (26.7%) fully reported more than half of the STARD items. The lowest number of fully reported items in a study was 5 and the highest was 17. Descriptions of key aspects of methodology frequently were missing. For example, details of participant sampling (e.g., consecutive or random selection) were described in only 8 (26.7%) of 30 publications. Measures of statistical uncertainty were reported in 18 (60%) of 30 publications. No single STARD item was fully reported by all the papers. Conclusions: The standard of reporting of diagnostic accuracy studies in glaucoma using OCT was suboptimal. It is hoped that adoption of the STARD guidelines will lead to an improvement in reporting of diagnostic accuracy studies, enabling clearer evidence to be produced for the usefulness of OCT for the diagnosis of glaucoma. © 2007 American Academy of Ophthalmology.

Sample size in studies on diagnostic accuracy in ophthalmology:A literature survey

Aim: To assess the sample sizes used in studies on diagnostic accuracy in ophthalmology. Design and sources: A survey literature published in 2005. Methods: The frequency of reporting calculations of sample sizes and the samples' sizes were extracted from the published literature. A manual search of five leading clinical journals in ophthalmology with the highest impact (Investigative Ophthalmology and Visual Science, Ophthalmology, Archives of Ophthalmology, American Journal of Ophthalmology and British Journal of Ophthalmology) was conducted by two independent investigators. Results: A total of 1698 articles were identified, of which 40 studies were on diagnostic accuracy. One study reported that sample size was calculated before initiating the study. Another study reported consideration of sample size without calculation. The mean (SD) sample size of all diagnostic studies was 172.6 (218.9). The median prevalence of the target condition was 50.5%. Conclusion: Only a few studies consider sample size in their methods. Inadequate sample sizes in diagnostic accuracy studies may result in misleading estimates of test accuracy. An improvement over the current standards on the design and reporting of diagnostic studies is warranted.

Diagnostic accuracy of pulmonary host inflammatory mediators in the exclusion of ventilator-acquired pneumonia

Background: Excessive use of empirical antibiotics is common in critically ill patients. Rapid biomarker-based exclusion of infection may improve antibiotic stewardship in ventilator-acquired pneumonia (VAP). However, successful validation of the usefulness of potential markers in this setting is exceptionally rare.

Objectives: We sought to validate the capacity for specific host inflammatory mediators to exclude pneumonia in patients with suspected VAP. 

Methods: A prospective, multicentre, validation study of patients with suspected VAP was conducted in 12 intensive care units. VAP was confirmed following bronchoscopy by culture of a potential pathogen in bronchoalveolar lavage fluid (BALF) at >10⁴ colony forming units per millilitre (cfu/mL). Interleukin-1 beta (IL-1β), IL-8, matrix metalloproteinase-8 (MMP-8), MMP-9 and human neutrophil elastase (HNE) were quantified in BALF. Diagnostic utility was determined for biomarkers individually and in combination. 

Results: Paired BALF culture and biomarker results were available for 150 patients. 53 patients (35%) had VAP and 97 (65%) patients formed the non-VAP group. All biomarkers were significantly higher in the VAP group (p<0.001). The area under the receiver operator characteristic curve for IL-1β was 0.81; IL-8, 0.74; MMP-8, 0.76; MMP-9, 0.79 and HNE, 0.78. A combination of IL-1β and IL-8, at the optimal cut-point, excluded VAP with a sensitivity of 100%, a specificity of 44.3% and a post-test probability of 0% (95% CI 0% to 9.2%). 

Conclusions: Low BALF IL-1β in combination with IL-8 confidently excludes VAP and could form a rapid biomarker-based rule-out test, with the potential to improve antibiotic stewardship.

Diagnostic accuracy of loop-mediated isothermal amplification as a near-patient test for meningococcal disease in children: An observational cohort study

Background: Diagnosis of meningococcal disease relies on recognition of clinical signs and symptoms that are notoriously non-specific, variable, and often absent in the early stages of the disease. Loop-mediated isothermal amplification (LAMP) has previously been shown to be fast and effective for the molecular detection of meningococcal DNA in clinical specimens. We aimed to assess the diagnostic accuracy of meningococcal LAMP as a near-patient test in the emergency department.

Methods: For this observational cohort study of diagnostic accuracy, children aged 0-13 years presenting to the emergency department of the Royal Belfast Hospital for Sick Children (Belfast, UK) with suspected meningococcal disease were eligible for inclusion. Patients underwent a standard meningococcal pack of investigations testing for meningococcal disease. Respiratory (nasopharyngeal swab) and blood specimens were collected from patients and tested with near-patient meningococcal LAMP and the results were compared with those obtained by reference laboratory tests (culture and PCR of blood and cerebrospinal fluid).

Findings: Between Nov 1, 2009, and Jan 31, 2012, 161 eligible children presenting at the hospital underwent the meningococcal pack of investigations and were tested for meningococcal disease, of whom 148 consented and were enrolled in the study. Combined testing of respiratory and blood specimens with use of LAMP was accurate (sensitivity 89% [95% CI 72-96], specificity 100% [97-100], positive predictive value 100% [85-100]; negative predictive value 98% [93-99]) and diagnostically useful (positive likelihood ratio 213 [95% CI 13-infinity] and negative likelihood ratio 0·11 [0·04-0·32]). The median time required for near-patient testing from sample to result was 1 h 26 min (IQR 1 h 20 min-1 h 32 min).

Interpretation: Meningococcal LAMP is straightforward enough for use in any hospital with basic laboratory facilities, and near-patient testing with this method is both feasible and effective. By contrast with existing UK National Institute of Health and Care Excellence guidelines, we showed that molecular testing of non-invasive respiratory specimens from children is diagnostically accurate and clinically useful.

Rapid detection of health-care-associated bloodstream infection in critical care using multipathogen real-time polymerase chain reaction technology: a diagnostic accuracy study and systematic review

Background: There is growing interest in the potential utility of real-time polymerase chain reaction (PCR) in diagnosing bloodstream infection by detecting pathogen deoxyribonucleic acid (DNA) in blood samples within a few hours. SeptiFast (Roche Diagnostics GmBH, Mannheim, Germany) is a multipathogen probe-based system targeting ribosomal DNA sequences of bacteria and fungi. It detects and identifies the commonest pathogens causing bloodstream infection. As background to this study, we report a systematic review of Phase III diagnostic accuracy studies of SeptiFast, which reveals uncertainty about its likely clinical utility based on widespread evidence of deficiencies in study design and reporting with a high risk of bias. 

Objective: Determine the accuracy of SeptiFast real-time PCR for the detection of health-care-associated bloodstream infection, against standard microbiological culture. 

Design: Prospective multicentre Phase III clinical diagnostic accuracy study using the standards for the reporting of diagnostic accuracy studies criteria. 

Setting: Critical care departments within NHS hospitals in the north-west of England. 

Participants: Adult patients requiring blood culture (BC) when developing new signs of systemic inflammation. 

Main outcome measures: SeptiFast real-time PCR results at species/genus level compared with microbiological culture in association with independent adjudication of infection. Metrics of diagnostic accuracy were derived including sensitivity, specificity, likelihood ratios and predictive values, with their 95% confidence intervals (CIs). Latent class analysis was used to explore the diagnostic performance of culture as a reference standard. 

Results: Of 1006 new patient episodes of systemic inflammation in 853 patients, 922 (92%) met the inclusion criteria and provided sufficient information for analysis. Index test assay failure occurred on 69 (7%) occasions. Adult patients had been exposed to a median of 8 days (interquartile range 4–16 days) of hospital care, had high levels of organ support activities and recent antibiotic exposure. SeptiFast real-time PCR, when compared with culture-proven bloodstream infection at species/genus level, had better specificity (85.8%, 95% CI 83.3% to 88.1%) than sensitivity (50%, 95% CI 39.1% to 60.8%). When compared with pooled diagnostic metrics derived from our systematic review, our clinical study revealed lower test accuracy of SeptiFast real-time PCR, mainly as a result of low diagnostic sensitivity. There was a low prevalence of BC-proven pathogens in these patients (9.2%, 95% CI 7.4% to 11.2%) such that the post-test probabilities of both a positive (26.3%, 95% CI 19.8% to 33.7%) and a negative SeptiFast test (5.6%, 95% CI 4.1% to 7.4%) indicate the potential limitations of this technology in the diagnosis of bloodstream infection. However, latent class analysis indicates that BC has a low sensitivity, questioning its relevance as a reference test in this setting. Using this analysis approach, the sensitivity of the SeptiFast test was low but also appeared significantly better than BC. Blood samples identified as positive by either culture or SeptiFast real-time PCR were associated with a high probability (> 95%) of infection, indicating higher diagnostic rule-in utility than was apparent using conventional analyses of diagnostic accuracy. 

Conclusion: SeptiFast real-time PCR on blood samples may have rapid rule-in utility for the diagnosis of health-care-associated bloodstream infection but the lack of sensitivity is a significant limiting factor. Innovations aimed at improved diagnostic sensitivity of real-time PCR in this setting are urgently required. Future work recommendations include technology developments to improve the efficiency of pathogen DNA extraction and the capacity to detect a much broader range of pathogens and drug resistance genes and the application of new statistical approaches able to more reliably assess test performance in situation where the reference standard (e.g. blood culture in the setting of high antimicrobial use) is prone to error.

STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies

Incomplete reporting has been identified as a major source of avoidable waste in biomedical research.
Essential information is often not provided in study reports, impeding the identification, critical
appraisal, and replication of studies. To improve the quality of reporting of diagnostic accuracy
studies, the Standards for Reporting Diagnostic Accuracy (STARD) statement was developed. Here
we present STARD 2015, an updated list of 30 essential items that should be included in every
report of a diagnostic accuracy study. This update incorporates recent evidence about sources of
bias and variability in diagnostic accuracy and is intended to facilitate the use of STARD. As such,
STARD 2015 may help to improve completeness and transparency in reporting of diagnostic accuracy
studies.